The method and system described in this patent specification relate to displaying radiological images and other information in a manner believed to assist their reading and interpretation by physicians or other users of the method and system. More specifically, the patent specification relates to a computer-aided diagnosis (xe2x80x9cCADxe2x80x9d) method and system for the detection and identification of anatomic abnormalities from radiological images viewed in conjunction with respective annotated road maps of the locations and/or characteristics of suspected abnormalities found through computer processing of the radiological images in accordance with the method and system described herein. The annotated maps highlight and/or identify characteristics of suspected abnormalities to help the user better assess the presence and/or meaning and significance of abnormalities in said radiological images.
The detection of suspected abnormal anatomic regions in radiological images using a computer system comprising specialized software and sometimes specialized hardware has been reported. For example, in the area of mammography, two books published in the last few years are: (1) xe2x80x9cDigital Mammography,xe2x80x9d edited by A. G. Gale et al, published by Elsevier in 1994; and (2) xe2x80x9cDigital Mammography ""96,xe2x80x9d edited by K. Doi et al, published by Elsevier in 1996. This subject matter has also been discussed in recently issued patents, for example: U.S. Pat. Nos. 4,907,156 (Doi et al); 5,133,020 (Giger et al), 5,212,637 (Saxena); 5,331,550 (Stafford et al); 5,365,429 (Carman); 5,452,367 (Bick et al); 5,491,627 (Zhang et al); 5,537,485 (Nishikawa et al); 5,572,565 (Abdel-Moottaleb); 5,574,799 (Bankman et al); 5,579,360 (Abdel-Moottaleb); 5,586,160 (Mascio); 5,598,481 (Nishikawa et al); 5,615,243 (Chang et al); 5,627,907 (Gur et al); 5,633,948 (Kegelmeyer); 5,661,820 (Kegelmeyer); 5,657,362 (Giger et al); 5,666,434 (Nishikawa et al); 5,673,332 (Nishikawa et al); 5,729,620 (Wang); 5,732,697 (Zhang et al); 5,740,268 (Nishikawa et al); and 5,815,591 (Roehrig et al). Additionally, a commercial mammographic CAD system is being sold in this country under the trade name xe2x80x9cImageCheckerxe2x80x9d by R2 Technology, Inc., a licensee of applicant. See also, the parent applications identified above, including the references cited therein as prior art or otherwise. See also, copending patent application Ser. No. 09/178,901, filed on Oct. 26, 1998 and entitled xe2x80x9cMethod and System for Computer-aided Lesion Detection Using Information from Multiple Imagesxe2x80x9d now U.S. Pat. No. 6,075,879. The two books cited earlier, as well and the earlier-cited patent applications and patents, including the references cited therein, are hereby incorporated by reference in this patent specification as though fully set forth herein.
In a screening radiological procedure, such as screening mammography, the patients typically are asymptomatic and cancers are said to be found at a typical rate of about one to ten cases per one thousand patient examinations. Reading the mammograms, when the vast majority of them are negative, can be a tedious task.
It has been reported that it is difficult for physicians to maintain a constantly high attention level. Some abnormalities that can be detected or diagnosed from the mammograms can be missed or misdiagnosed, which can result in delayed or more costly treatment, and can even result in a reduction of a patient""s longevity or chance of survival. According to an article in the May 26,1993 issue of JAMA, pages 2616-2617, the misdiagnosis rate in mammograms can be in the range of 15 to 63%. Several mammography clinical studies, as summarized in an article entitled xe2x80x9cEstimating the accuracy of screening mammography: a meta-analysis,xe2x80x9d published in Am. J. Prev. Med. (1998), volume 14, pages 143-152, indicate that the false negative (missed cancer) rate ranges from 5% to 17% for women""s first screening and increases to 14% to 44% for subsequent screenings. The CAD system, serving as an electronic reminder or second reader, or as a spell-checker can be in a word processor, can assist physicians in attaining higher detection rate (higher sensitivity) for cancers or in reducing the misdiagnosis rate (lowering the false-negative rate). Similar considerations apply to radiological imaging other than mammography.
Applicant understands that a current procedure using a commercial CAD system is as follows. The radiological technician or the physician takes a set of radiological images of the patient following a preprogrammed protocol or procedure. A physician views the radiological images and reaches a preliminary diagnostic decision. The physician next views separate, second images generated by said CAD method and system. Each second image is a copy or a sub-sampled (with fewer pixels) copy of a respective radiological image, marked or annotated with a localized notation of the suspected abnormalities the CAD system has detected through computer analysis of a digitized version of the respective radiological image. After any reexamination of the areas of the radiological images that correspond to the positions of the suspected abnormalities displayed on the CAD system, the physician makes a final diagnostic decision. The final diagnostic decision may or may not be the same as the preliminary decision, depending on whether the physician found the additional information provided by the CAD system to be significant and, if so, what significance the physician ascribed to it. Following the final diagnostic decision, and perhaps depending on the degree of suspicion for malignancy, the physician can recommend a course of further action, which can include no action for at least some period or time, or further follow-up examinations, or biopsy.
The following describes a typical procedure for detecting suspected abnormal anatomic features in mammograms using the commercial mammographic CAD system sold under the trade name ImageChecker. A set of four x-ray film mammograms, two views of each breast of a patient, taken by a radiological technician or a physician, are processed through a film digitizer to generate a set of four digitized images that are input as such into the ImageChecker system. Each of the four digitized images is then analyzed by a digital image processing computer with specialized software and typically some specialized hardware as well. The processing detects anatomic features that meet criteria for suspected abnormalities, and creates a respective annotated image for each of the film mammograms. The original film mammograms are then mounted on a conventional lightbox or a motorized lightbox, an alternator, for viewing. The annotated images of these mammograms are displayed on two small TV monitors located beneath the lightbox. Each small monitor displays two annotated images. Each annotated image comprises a sub-sampled digitized image of the respective film mammogram and locational markers marking the locations of the suspected abnormalities that the CAD processing detected. Currently, two different markers convey information regarding two key cancerous features of the suspected abnormalities that are detected. One marker is a triangle-shaped marker used to mark the location of a suspected abnormal cluster of microcalcifications. The other marker is a star-shaped marker used to mark the location of a suspected abnormal mass.
During a mammographic screening procedure, the CAD system typically overmarks suspected abnormalities in order to ensure a high detection rate. Currently, the CAD systems in commercial use tend to report suspected abnormalities with relative probability values above a certain selected threshold, and at a rate much higher, perhaps a hundred times higher, than the rate at which a physician would judge the abnormality sufficiently suspicious to warrant a recall of the patient for additional diagnostic work-up examinations. Therefore, a physician may have to examine and perhaps dismiss not only the suspected abnormalities that a physician would detect from the film mammograms but also the typically greater number of suspected abnormalities detected by the CAD system. Depending on how many more additional suspected abnormalities the CAD system detects and identifies on the display, the extra task of dismissing them can slow down the reading and assessment process.
During a mammographic examination, the radiological technician sometimes takes extra views of a patient if the technician suspects or believes that an abnormality exists, or if a technical error is likely to have degraded a mammogram (such as patient motion, exposure error, positioning error, etc.). The added information from these extra views is believed to have the potential of enabling the physician to reduce recall rates by assisting in assessing/dismissing abnormalities. However, since some abnormalities are difficult to detect, sometimes even to a physician, there is little assurance that the technician can consistently detect and take extra views of these abnormalities.
An object of the method and system described in this patent specification is to provide annotation information that can include an assessment of the probability, likelihood or predictive value of the CAD-detected suspected abnormalities as an additional aid to the radiologist or other user of the method and system. It is believed that an improvement in the display of the CAD information, by adding information such as probability information to the locational markers of the CAD-detected suspected abnormalities, would make it easier for the physician in assessing/dismissing the CAD-detected markers. A more specific object is to display a value in numerical form representing the probability, likelihood or predictive values of the CAD-detected suspected abnormalities, next to the locational markers of the CAD-detected suspected abnormalities. Another object is to use the locational markers of CAD-detected suspected abnormalities to display a value such as probability of the CAD-detected suspected abnormalities in analog form by varying the shape, size and/or the color of the locational markers. Still another object is to display only a few, e.g., two or three, different (in shape, color, etc.) locational markers for each type of the CAD-detected suspected abnormality (such as clusters of microcalcifications, mass, etc.), wherein each different marker represents a different threshold of information such as probability. Still another object is to provide a display of so-called xe2x80x9cextra view markersxe2x80x9d as an additional aid or guide to the radiologic technician so that extra views may be taken of the patient before the patient is released or discharged. It is believed that with the added information from these extra views, the physician can perform the examination with improved accuracy and that this can result in a reduced patient recall rate. Yet another object is to provide a control device to allow the user to select the threshold for information such as probability for the display of locational markers.
In a first exemplary and non-limiting embodiment of the method and system described in this patent specification, probability values in numerical form, representing the probability, likelihood and/or predictive values of the CAD-detected abnormalities, are displayed next to the locational markers of the CAD-detected suspected abnormalities on the annotation maps. In a second exemplary embodiment, the locational markers of CAD-detected suspected abnormalities display probability values of the CAD-detected suspected abnormalities in analog form by varying the shape, size or some other visual characteristic color of these locational markers. In a third exemplary embodiment, two different locational markers representing two different selected probability thresholds are displayed on an annotated map of the CAD-detected abnormalities. In a fourth embodiment, so-called xe2x80x9cextra view markersxe2x80x9d, representing a selected probability threshold, are displayed as an additional aid to the radiologic technician so that extra views may be taken of the patient before the patient is released or discharged. In a fifth exemplary embodiment, a control device is provided to allow the user to select and vary the probability thresholds for the display of locational markers.
Stated in broader terms, one aspect of the method and system described herein is to provide annotation information that can include an assessment of the probability, likelihood or predictive value of the CAD-detected abnormalities as an additional aid to radiologists, radiologic technicians, or other users. While detailed examples of mammography applications are described, it should be clear that the novel method and system apply to other radiological images as well, such as projection x-ray images of other body parts, CT images, MRI images, etc.